Proactive Security Testing and Fuzzing

Software is bound to have security critical flaws, and no testing or code auditing can ensure that software is flawless. But software security testing requirements have improved radically during the past years, largely due to criticism from security conscious consumers and Enterprise customers. Whereas in the past, security flaws were taken for granted (and patches were quietly and humbly installed), they now are probably one of the most common reasons why people switch vendors or software providers. The maintenance costs from security updates often add to become one of the biggest cost items to large Enterprise users. Fortunately test automation techniques have also improved. Techniques like model-based testing (MBT) enable efficient generation of security tests that reach good confidence levels in discovering zero-day mistakes in software. This technique is called fuzzing. 1 Reactive versus Proactive Security has traditionally been reactive, focused on defending from attacks. A proactive approach should focus on fixing the actual flaws enabling these attacks. An attack does not work if there is no vulnerability. Majority of flaws reported publicly are found by third parties, and require expensive and timesensitive process for disclosure of the vulnerability data, building of corrections (patches) and distribution/deployment of the corrective measures. A bug found as part of the software development process will not go through this extensive process but is handled just like any other critical flaw in the system. 1.1 Fuzzing as a Proactive Measure Proactive security testing approaches include fuzzing, protocol mutation, robustness testing, and the like. Especially fuzzing is a very effective way of discovering software vulnerabilities, as it requires no intelligence of the internal operations of the device or system under test. Legacy fuzzing was based on randomly mutated inputs (white-noise testing), and was only used by professional security specialists and selected researchers. But today most fuzzers are based on intelligent model-based test automation techniques. Besides making the use of fuzzing tools much easier, this enables much higher vulnerability detection rates. 1.2 Automation equals Efficiency Fuzzing is a rather new test automation technique for finding critical security problems in any type of communication software. It is a negative software testing method (negative testing) that feeds a program, device or system with malformed and unexpected input data in order to find critical crash-level defects. The tests are targeted at remote interfaces, but can also test local interfaces and API. Focus on most critical remote interfaces typically means that fuzzing is able to cover the most exposed and critical attack surfaces in a system relatively well, and identify many common errors and potential vulner2 Proactive Security Testing and Fuzzing abilities quickly and cost-effectively. Only recently, it was mostly an unknown hacking technique that very few quality assurance specialists knew about. Today, both quality assurance engineers and security auditors use fuzzing. It is a mainstream testing technique used by all major companies building software and devices for critical communication infrastructure. 1.3 Focus on Finding Vulnerabilities Fuzzing is focused on detecting implementation issues in software. Vulnerability databases indicate that programming errors causes 80% of the publicly known vulnerabilities. Inclusion of the vulnerabilities caught in the software development would probably increase this even further. Today, no more than 25% of vulnerabilities are found with the traditional software quality assurance processes, with majority of the software companies catching less than 5% of the vulnerabilities in hiding. Static analysis tools cannot be used in post-release testing, or in third party security analysis. Improving the software testing practices can eliminate these worm-size holes before product launch, and without requiring access to the source code. 1.4 Evolution of Fuzzing The term ‘fuzzing’ or ‘fuzz testing’ emerged around 1990, but in its original meaning fuzzing was just another name for random testing, with very little use in Quality Assurance (QA) beyond some limited ad-hoc testing. Still, the transition to integrating the approach into software development was evident even back then. During 1998-2001, in the PROTOS project (at University of Oulu) we conducted research that had a focus on new model-based test automation techniques, and other next generation fuzzing techniques [Kaks02]. The purpose was to enable the software industry themselves to find security critical problems in a wide range of communication products, and not to just depend on vulnerability disclosures from third parties. Codenomicon is a spin-off from the project, founded in 2001, that today continues to lead the state-of-the-art in fuzzing techniques. Later, around year 2007 also other companies became fascinated in the topic, and a new highly competed test and measurement market domain was created. Finally in 2008, among several other books on the topic, we also released a book (with the help of other fuzzing specialists) which gives a broader and but also more detailed look on how fuzzing can be used in different steps in the software lifecycle [TaDM08]. This started the move towards making fuzzing a best practise in software development.